Screen content, or data describing information displayed to a user by a computing system on a display, generally includes a number of different types of content. These can include, for example, text content, video content, static images (e.g., displays of windows or other GUI elements), and slides or other presentation materials. Increasingly, screen content is delivered remotely, for example so that two or more remote computing systems can share a common display, allowing two remotely-located individuals to view the same screen simultaneously, or otherwise in a teleconference such that a screen is shared among multiple individuals. Because screen content is delivered remotely, and due to increasing screen resolutions, it is desirable to compress this content to a size below its native bitmap size, to conserve bandwidth and improve efficiency in transmission.
Although a number of compression solutions exist for graphical data such as screen content, these compression solutions are inadequate for use with variable screen content. For example, traditional Moving Picture Experts Group (MPEG) codecs provide satisfactory compression for video content, since the compression solutions rely on differences between sequential frames. Furthermore, many devices have integrated MPEG decoders that can efficiently decode such encoded data. However, MPEG encoding does not provide substantial data compression for non-video content that may nevertheless change over time, and therefore is not typically used for screen content, in particular for remote screen display.
To address the above issues, a mix of codecs might be used for remote delivery of graphical data. For example, text data may use a lossless codec, while for screen background data or video data, a lossy codec that compresses the data may be used (e.g., MPEG-4 AVC/264). Additionally, in some cases, the lossy compression may be performed on a progressive basis. However, this use of mixed codecs raises issues. For example, there is often a difference in visual quality when comparing portions of screen content that use lossy and lossless codecs. Additionally, in a typical lossy codec, quantization is used to improve compression at a loss to quality. It is common to quantize higher frequency content more than lower frequency content. This results in loss of sharpness and introduction of quantization noise, in the reconstructed image, which translates to visible artifacts in the reconstructed image. This problem is more acute at higher quantization levels or lower bandwidths. In particular, when bandwidth is insufficient, lossy codecs rapidly lose visual quality, in particular progressively updated codecs. This can introduce substantial jitter into the image.